When training early-stage deep neural networks (DNNs), generating intermediate features via convolution or linear layers occupied most of the execution time. Accordingly, extensive research has been done to reduce the computational burden of the convolution or linear layers. In recent mobile-friendly DNNs, however, the relative number of operations involved in processing these layers has significantly reduced. As a result, the proportion of the execution time of other layers, such as batch normalization layers, has increased. Thus, in this work, we conduct a detailed analysis of the batch normalization layer to efficiently reduce the runtime overhead in the batch normalization process. Backed up by the thorough analysis, we present an extremely efficient batch normalization, named LightNorm, and its associated hardware module. In more detail, we fuse three approximation techniques that are i) low bit-precision, ii) range batch normalization, and iii) block floating point. All these approximate techniques are carefully utilized not only to maintain the statistics of intermediate feature maps, but also to minimize the off-chip memory accesses. By using the proposed LightNorm hardware, we can achieve significant area and energy savings during the DNN training without hurting the training accuracy. This makes the proposed hardware a great candidate for the on-device training.

Flow-guide synthesis provides a common framework for frame interpolation, where optical flow is typically estimated by a pyramid network, and then leveraged to guide a synthesis network to generate intermediate frames between input frames. In this paper, we present UPR-Net, a novel Unified Pyramid Recurrent Network for frame interpolation. Cast in a flexible pyramid framework, UPR-Net exploits lightweight recurrent modules for both bi-directional flow estimation and intermediate frame synthesis. At each pyramid level, it leverages estimated bi-directional flow to generate forward-warped representations for frame synthesis; across pyramid levels, it enables iterative refinement for both optical flow and intermediate frame. In particular, we show that our iterative synthesis can significantly improve the robustness of frame interpolation on large motion cases. Despite being extremely lightweight (1.7M parameters), UPR-Net achieves excellent performance on a large range of benchmarks. Code will be available soon.

We propose an end-to-end music mixing style transfer system that converts the mixing style of an input multitrack to that of a reference song. This is achieved with an encoder pre-trained with a contrastive objective to extract only audio effects related information from a reference music recording. All our models are trained in a self-supervised manner from an already-processed wet multitrack dataset with an effective data preprocessing method that alleviates the data scarcity of obtaining unprocessed dry data. We analyze the proposed encoder for the disentanglement capability of audio effects and also validate its performance for mixing style transfer through both objective and subjective evaluations. From the results, we show the proposed system not only converts the mixing style of multitrack audio close to a reference but is also robust with mixture-wise style transfer upon using a music source separation model.

Drowsiness on the road is a widespread problem with fatal consequences; thus, a multitude of systems and techniques have been proposed. Among existing methods, Ghoddoosian et al. utilized temporal blinking patterns to detect early signs of drowsiness, but their algorithm was tested only on a powerful desktop computer, which is not practical to apply in a moving vehicle setting. In this paper, we propose an efficient platform to run Ghoddosian's algorithm, detail the performance tests we ran to determine this platform, and explain our threshold optimization logic. After considering the Jetson Nano and Beelink (Mini PC), we concluded that the Mini PC is the most efficient and practical to run our embedded system in a vehicle. To determine this, we ran communication speed tests and evaluated total processing times for inference operations. Based on our experiments, the average total processing time to run the drowsiness detection model was 94.27 ms for Jetson Nano and 22.73 ms for the Beelink (Mini PC). Considering the portability and power efficiency of each device, along with the processing time results, the Beelink (Mini PC) was determined to be most suitable. Also, we propose a threshold optimization algorithm, which determines whether the driver is drowsy or alert based on the trade-off between the sensitivity and specificity of the drowsiness detection model. Our study will serve as a crucial next step for drowsiness detection research and its application in vehicles. Through our experiment, we have determinend a favorable platform that can run drowsiness detection algorithms in real-time and can be used as a foundation to further advance drowsiness detection research. In doing so, we have bridged the gap between an existing embedded system and its actual implementation in vehicles to bring drowsiness technology a step closer to prevalent real-life implementation.

在半导体制造中，晶圆地图缺陷模式为设施维护和产量管理提供了关键信息，因此缺陷模式的分类是制造过程中最重要的任务之一。在本文中，我们提出了一种新颖的方式来表示缺陷模式作为有限维矢量的形状，该矢量将用作分类神经网络算法的输入。主要思想是使用拓扑数据分析（TDA）的持续同源性理论提取每种模式的拓扑特征。通过使用模拟数据集进行的一些实验，我们表明，与使用卷积神经网络（CNN）的方法相比，所提出的方法在训练方面更快，更有效地训练，这是晶圆映射缺陷模式分类的最常见方法。此外，当训练数据的数量不够并且不平衡时，我们的方法优于基于CNN的方法。

大规模的语音自我监督学习（SSL）已经出现到语音处理的主要领域，但是，由于其巨大规模而引起的计算成本问题是对学术界的高障碍。此外，语音SSL模型的现有蒸馏技术通过减少层来压缩模型，从而在语言模式识别任务（例如音素识别（PR））中引起性能降解。在本文中，我们提出了Fithubert，它几乎在几乎所有模型组件中都使尺寸较薄，并且与先前的语音SSL蒸馏作品相比，层层更深。此外，我们采用缩短时间来加快推理时间，并提出一种基于提示的蒸馏方法，以减少性能降解。与休伯特相比，我们的方法将模型降低到23.8％，推理时间为35.9％。此外，我们在优越的基准上达到了12.1％的单词错误率和13.3％的音素错误率，这比先前的工作优越。

我们为基于运动的视频框架插值提供了一种新颖的简单而有效的算法。现有的基于运动的插值方法通常依赖于预先训练的光流模型或基于U-NET的金字塔网络进行运动估计，该运动估计要么具有较大的模型大小或有限的处理复合物和大型运动案例的容量。在这项工作中，通过仔细整合了中间方向的前射击，轻质特征编码器和相关量为金字塔复发框架，我们得出一个紧凑的模型，以同时估计输入帧之间的双向运动。它的尺寸比PWC-NET小15倍，但可以更可靠，更灵活地处理具有挑战性的运动案例。基于估计的双向运动，我们向前射击输入帧及其上下文特征到中间帧，并采用合成网络来估算扭曲表示的中间帧。我们的方法在广泛的视频框架插值基准测试中实现了出色的性能。代码将很快可用。

了解多媒体内容中描述或显示的事件彼此相关是开发可用于真实世界媒体的强大人工智能系统的关键组成部分。尽管许多研究专门用于文本，图像和视频域中的事件理解，但没有一个研究探索事件跨域中经历的复杂关系。例如，新闻文章可能会描述“抗议”事件，而视频显示“逮捕”事件。认识到视觉“逮捕”事件是更广泛的“抗议”事件的一个子事件，这是一个具有挑战性但重要的问题，但前面的工作尚未探讨。在本文中，我们提出了多模式事件关系关系的新任务，以识别这种跨模式事件关系。我们贡献了一个大规模数据集，该数据集由100K视频新文章对组成，以及密集注释的数据的基准。我们还提出了一种弱监督的多模式方法，该方法将来自外部知识库（KB）的常识性知识整合在一起，以预测丰富的多模式事件层次结构。实验表明，我们的模型在我们提出的基准上优于许多竞争基线。我们还对模型的性能进行了详细的分析，并建议未来研究的方向。

组织病理学仍然是各种癌症诊断的黄金标准。计算机视觉的最新进展，特别是深度学习，促进了针对各种任务的组织病理学图像的分析，包括免疫细胞检测和微卫星不稳定性分类。每个任务的最新工作通常采用鉴定的基础体系结构，这些体系结构已鉴定为图像分类。开发组织病理学分类器的标准方法倾向于将重点放在优化单个任务的模型上，而不是考虑建模创新的各个方面，从而改善了跨任务的概括。在这里，我们提出了Champkit（模型预测工具包的全面组织病理学评估）：可扩展的，完全可重现的基准测试工具包，由大量的斑点级图像分类任务组成，跨不同的癌症。 Champkit能够系统地记录模型和方法中提议改进的性能影响的一种方法。 Champkit源代码和数据可在https://github.com/kaczmarj/champkit上自由访问。

基于深度学习的技术为自动图像质量评估（IQA）领域的显着进步做出了贡献。现有的IQA方法旨在根据图像级别（即整个图像）或贴片级（将图像分为多个单元和测量每个图像的质量在图像级别（即整个图像）处的平均意见分数（MOS）来衡量图像的质量修补）。某些应用可能需要评估像素级别（即每个像素的MOS值）处的质量，但是，由于其网络结构而丢失了空间信息，因此在现有技术的情况下不可能评估这是不可能的。本文提出了一种IQA算法，除图像级MOS外，还可以测量像素级的MOS。提出的算法由三个核心部分组成，即：i）本地IQA； ii）感兴趣的区域（ROI）预测； iii）高级功能嵌入。本地IQA部件在像素级或像素MOS上输出MOS - 我们称其为“ PMOS”。 ROI预测部分输出的权重来计算图像级IQA时区域的相对重要性。嵌入零件的高级特征提取高级图像特征，然后将其嵌入到本地IQA部分中。换句话说，提出的算法产生三个输出：代表每个像素的MOS的PMO，来自ROI的权重表示区域的相对重要性，最后是通过PMOS和ROI加权总和获得的图像级MOS值。与现有流行的IQA技术相比，通过使用PMO和ROI权重获得的图像级MOS表现出较高的性能。此外，可视化结果表明，预测的PMO和ROI输出与人类视觉系统（HVS）的一般原理相当一致。